Fast Weight Long Short-term Memory

نویسندگان

  • T. Anderson Keller
  • Sharath Nittur Sridhar
  • Xin Wang
چکیده

Associative memory using fast weights is a short-term memory mechanism that substantially improves the memory capacity and time scale of recurrent neural networks (RNNs). As recent studies introduced fast weights only to regular RNNs, it is unknown whether fast weight memory is beneficial to gated RNNs. In this work, we report a significant synergy between long short-term memory (LSTM) networks and fast weight associative memories. We show that this combination, in learning associative retrieval tasks, results in much faster training and lower test error, a performance boost most prominent at high memory task difficulties.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

The effect of intrahippocampal microinjection of Naloxone on short –term and long-term memory in adult male rats

Introduction:The hippocampus is one for the major centers of learning and memory. Role of the opioid system has been investigated and on the other hand receptors related to this system such as mu-opioid receptors (MOR) are extended in the hippocampus. In this study the effect of Naloxone administration as a mu opioid receptor antagonist on passive avoidance memory in adult male rats was i...

متن کامل

Effects of fresh, aged and cooked garlic extracts on short- and long-term memory in diabetic rats

Objective: The present study was hypothesized to investigate the beneficial effects of fresh, aged, and cooked garlic extracts on blood glucose and memory of diabetic rats induced by streptozocine (STZ). Material and Methods: Diabetes was induced by an intraperitoneal injection of STZ (60 mg/kg body weight). An oral dose of 1000 mg/kg of each garlic extract was given daily for 4 weeks after dia...

متن کامل

Effects of exercise on spatial memory deficits induced by nucleus basalis magnocellularis lesions

Introduction: Previous studies have shown that exercise enhances cognitive and functional capacities in patients with Alzheimer's disease (AD). In this study, we investigated the effect of long-term (60 days) and short- term (10 days) exercise on the spatial memory deficits in an animal model of AD. Methods: Fifty male rats were divided into 5 groups 1) intact, 2) sham, 3) sham-Alzheimer 4) ...

متن کامل

Gated Fast Weights for Associative Retrieval

We improve previous end-to-end differentiable neural networks (NNs) with fast weight memories. A gate mechanism updates fast weights at every time step of a sequence through two separate outer-product-based matrices generated by slow parts of the net. The system is trained on a complex sequence to sequence variation of the Associative Retrieval Problem with roughly 70 times more temporal memory...

متن کامل

Effect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia

   Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2018